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ABSTRACT 



. An examination of definitidn, causes, variables, and 

strategies of implementation suggests the kinds of data. necessary for 
measuring school effectiveness. By focusing on students who first 
enter school below grade level standards and by understanding that 
student learning is embedded in a system of key elements or variables 
that, V over time, are interrelated in a circular pattern of causality, 
longitudinal research provides such data on effective schooling. * 
Essential structural differences between effective and ineffective 
schools are highlighted by causal loop diagrams illustrating both key 
variables and the concept of circular causality. The variablei^, which 
identify areas requiring further empirical research, are teacher * 
effectiveness and expectations, time allocated to instruction,- 
engaged learning time, and^ student motivation, behavior, and 
achievement. The variabl^s;^ identify points of intervention la- the 
system where the principal's leadership role is. critical in achieving 
an effective school. After the problem hias been defined, causes 
understood, and pointi^ of intervention identified, a thfeory of* 
implementation is required. The' authors thus emphasize the need 
research on the organizational dynamic^s of implementing school 
effectiveness programs. (PB) 
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MEASURING SCHOOL' EFFECTIVENESS: A 'SYSTEMS PERSPECTIVE 



Karl H. Clauset, Jr... 
Aian K. Gaynor 
Boston uni^ve^slty 



The opening ' remarks ' for ':this Symposium highlight the current 
dilfitimas in measuring school effectiveness. These ^ stem from the two 
central issues in 'measurement — what 'to measure, and how to measure it. 
Thus, the purpose of this • symposium is to' focus attention on each of 
these issues in hopes "of resolving some of > the dilemmas^ in the research 
and practice of school e'ff ectiveness. • ^ ' • . 



perspective 



our own wor)c, atoout what 



We shall speak, from the 
em;airical data are required to advance theoretical understandings about 
.effective schboling and to support the successful implementation of 
sthool improvement programs. We ^pea)c as theoreticians whose effort.s 
have involved knowledge synthesis, theory, building, comput-er simulation 
modeling, afLtl policy analysis. We have approached the question of what 
to measure by examining four related questions in four* different areas: 



THE -NATURE OF THE PROBLEM 



What is an effective or ineffective 
school and what data must one have to 
assess the relative effectiveness of 
a particular school? « 



THE CAUSES OF THE PROBLEM 



What are the limitations in the 
literature with respect to 
understanding why schools are 
persistently ineffective ' and what 
data collection aftd analysis are 
needed? * • * 



"ALTERABLE" VARIABLES 



STRATEGIES FOR IMPLEK^TATION 



What are l:he most potent points of 
intervention for improving ' schools 
and what are the • most appropriate 
kinds of baseline data to collect? 

What •are the -most significant factors 
that interact to enhance or inhibit 
the implementation of school 
improvement programs and what data 
are necessary to ^measure the current 
levels of these variables? 
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. These questions' raise broader issues of definition, theory, and 
fact — issues which are closely intertwined. Definitions are rooted in 
values. Theories configure define* concepts according to logical rules 
and icnown facts. Facts derive from empirical research which, in turn, 
relies for; direction upon existing theory. Facts also depend on the 
science of measurement, rooted in epistemology and technology. Methods 
of research design and data analysis represent procedural manifestations 



of the logic of knowing. 

* • ; ^ 

^ It 4s not our intent here to focus discussion on these broader 
dimensions of the problem of ineffective schooling. Rather, we 
acknowledge that our perspective on these meta-issues shapes our 
thinking and our conclusions. In the pages that follow, we shall outline 
oUr own thinking about each one of the four questions put forth above 
and try to make our^ as sumptions clear. The central focus of our remarks 
will be on what data are necessary for ' measurement. We will leave- to 
others whose work focuses more directly on these matters discussioij of 
the technical issues of how to. measure. 

, ' defin ing the Problem 

'The first question we shall address relates* to how one defines the 
problem of school effectiveness. We ^ have adopted a systems perspective 
in approaching the issue of definition. Taking this perspective means 
that we view problems in organizations as problems that are 
\discrepancies\xver time between actjial organisational behaviors and some 
desired set of behaviors. For eyCample, in* studies by the Connecticut 
School * Effectiveness Project of less effective schools, thi desired 
behavior is that the same fraction of middle-class and disadvantaged 
children score above the 3pth percentile on standardized math and 
reading achievement tests; /the actual behavior is that the fraction of 
disadvantaged children is far lower; and the problem discrepancy is the 
gap between what the value of the fraction is and should be. It is this 
discrepancy that the State of Connecticut is trying .to" close . 

One of the issues in defining a problem in school jetffectiveness is 
deciding what the desired behaviors shpuld be and, therefore, what the 
problem discrepancy is. This is not an easy task since the decision is 
ultimately rooted in values. The second issue revolves around how one 
looks at the discrepancy. Ir is important to recognize that a problem in 
student achievement patterns is a longitudinal problem. An ineffective 
school does not become ineffective instantaneously. The ropts of 
ineffectiveness cannot be traced to an isolated event. We see the 
problem as bteing generated over time by the structure of the system. It^ 
is a systemic problem. That is, patterns in student achievement are the 
result of patterns of student-teacher interactions that tend to become 
established when a child enters the school and then persist from grade 
to grade. In an ineffective schoc^, if a child starts out below grade 
level, he gets labelled a "low achiever" and patterns of instruction are 
set in place that result in his falling further and further behind as tie 
moves from grade 1:0 grade. ^ The discrepancy measured in the sixth grade 
has its roots in patterns established in the first grade. 

So' our systrjms perspective on school effectiveness problems is that 
these problems are longitudinal in nature and that^ they are defined in 
terms of discrepancies between" what is and ^what should be. If 
researchers and practitioners agree that the problem is Jundamentally a 
longitudinal problem, then we seriously question the use of essentially" 
'Static and cross-sectional measures^ of school achievement — whetlher 
they be average scores, average gains, average passing rates, or these 
metrics differentiated by group. 



By contrast^ in our own research,- we formulated what might be called 
the "radioactive tracer" technique for measuring school effectiveness. 
Radioactive tracers are used in biology and medicine to monitor the 
functioning of an organism over time. The tracers are injected into the 
organism and, at selected time intervals, the organism is scanned with a 
radiation detector to determine 'where the tracers are. Analysis of the 
data over a number of time intervals reveals longitudinal patterns, that^ 
provide insigiits into the relative health^ of the organism or the' 
functioning of a particular part of the organism. 

In our. school effectiveness research, the organism is the school, 
the tracers are selected groups of students, and the radiation, detectors 
take the form of measures for tracking changes over time in behavior, 
motivation, instruction, and learning. A school is perceived as 
effective to the extent that the achievement pattern for eahh tracer 
group equals^ or exceeds the normal achievelnent* fiattern^ (i.e., Students 
•being on grade level and gaining at least one year "per year) . 

For us, the foc^ group of concern is comprised of those students 
who are, when they first enter school, initially below grade level 
standards. We fodlis on these students because the literature shows that 
a disproportionate number of thfese students are poor and/or minority 
children and because our .research shows that less effective Schools tend 
.to reinforce these initial achievement patterns. Thus, as students move 
from grade to 'grade , the initially high achi^;ters continue to excel, the 
initially average achievers continue to do average worlc, and the 
initially low achievers fall further and further behind grade level 
standards. By contrast in the effective school, the gap between grade 
level standards and the achievement ^pattern for initially low -achievers 
narrows and closes as the students move through school. 

It must be said that the current slate of knowledge about effective 
schooling is a strange one, indeed. There is general agreement that the 
problem of the widening*^ gap in ' learning between grade, level standards- 
and initially low-achieving students is a credible ' reality in. most 
schools. What we have not seen,, however , is solid documentary evidence 
to describe the precise nature of these trends in schools. What is still 
needed are widespread longitudinal studies of children in diverse school 
settings who are identified- on the basis of initial »• school entry 
characteristics and who are followed systematically over a jiumber of 
years of schooling. It is important to describe not only the shape of 
achievement patterns by cohort but also the. persistence of such patterns 
in particular schools, over, significant periods of time.* We need to know 
more about the, magnitude of these learning gaps and their variations 
within and across schools over time. Thus, in the broadest sense, we 
recognize that it is essential to have available "through empirical 
research the longitudinal measures by which the problem, itself, can be 
documented. 
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Cleaply, this data condition needs to ^e^rectified in large scale 
studies geograpnically dispersed across the country. Some efforts, in 
•the .State of Connecticut for example , are being made to implement the 
requisite studies and to develop appropriate procedures and instruments, 
but we believe that many systematic efforts need .to be ^'unded in many 
states. Undoubtedly, the sharing of approaches to instrumentation is 
crucial if these efforts , are to be carried out in wayi that are 
efficient and corapara*tive. [l] 

Undferstandlng the Causes of the Problem ^ . - 

A systems perspective thkt begins' with defining organizational 
problems as discrepancies that persist over time naturally leads to 
asking why these problems persist. Problems of this type are unlikely to 
h^e their root causes in isolated external even'ts. Rather, their 
causes are rooted in the structure of the system. Understanding why a 
problem ""persists means, for us, understanding the structure of the 
problem system^ — a system which might enconlpass both "organisational" 
and, "environmental" variables. [2] • ' 

The second question evolves from efforts to gaii^ atv understanding of 
why, schools are persistently ineffective^^: 'effebtive^. ^ In our own 
research we reviewed the current' literatuTel*^ in order to construct a 
causal theory'of school' effectiveness .and to translate that theory into 
a computer simulation model for policy analysis. In the process, we 
found areas in the literature where research findings were^ either 
contradictory or non-existent. We believe that further empiriccvl 
research is needed in these areas to help illuminate and advance, a 
theoretical' understanding of the dynamics of school effectiveness. Th^ 
following paragraphs briefly summar^e the theory and then address the' 
implications for research - that we We arising from our efforts in 
theory-building and modeling. ^V' 

B ackground . In the school effectiveness problem, we see the 
structure of the system as being a set of key variables (e.g., time 
allocated to ilnstruction> engaged time, teacher expectations, teacher 



[l] The Northeast Educational Exchange (NEREX) is currently supporting 
multiple efforts in the Northeastern states in the form of what are 
being called School Improvement-' Institutes. It is the purpose of 
these institutes to facilitate collaboration among local ' school 
f districts, universities, an^ other agengies for school improvement. 
These institutes are supporting, among other things, the 
development, testing, and transfer of procedures and instruments 
for evaluating various dimensions , of effective teaching and 
schooling. This effort is illustrative of the kinds of efforts 
proceeding nationally with respect to improving the effectiveness 
of schools for all kinds of students. 

[2] On Friday, April 1-5, at Session 57.09, we are presenting another 
paper which directly addresses this issue. The paper is entitled 
"Organizations and Their Environments: A System Dynamics 
Perspective." 
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effectiveness, student motivation, student behavior, and student 
achievement) that mutually interact- day-in and day-out through a network 
of causal relationships to produce the patterns in student achievement 
and school processes that are characterized as more or less effective. 

The essential systems idea here is that student learning is embedded 
in a system of elements whicjif over successive periods of time, are 
interrelated in circular patterns of causality. Thus, student 
achievement is not simply a "dependent" variable but an element in a 
network of circular causality. It is affected by other variables but it 
also has a retur n effect on those variables. Instruction, motivation, 
and behavior^ respond 'to variations in student achievement- just as the 
contrary is true. [ . 

This idea of circular causality differs from most traditional 
research which has tried to discover what factors . "affect student 
learning." The imp/Lication in traditional research has been that these 
"independent" variables cause variations in student learning, which 
typically has been perceived as the "dependent" variable. Causation is 
assumed to be in one direction only and path models are characteristic 
representations of this way of thinking. 

The Theory . Our approach to developing a ^theory of school 
effectiveness has been to construct a circular theory of causality that 
links student variables with organizational and instructional variables. 
The purpose of this * section i* to present a brief Overview of this 
theory of school effectiveness as a context for discussing the 
deficiencies found .in the literature and making' some suggestions 
a^out the kinds of data, that might be collected to erase these 
deficiencies. [3] The diagrams presented below, called "Causal Loop 
Diagrams," graphically ill^ustrate the key variables and the concept of 
circular causality. They also highlight the essential structural 
differences between schools that are effective and ineffective for 
initially low-achieving children. 
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[3] The following works offer a more detailed explication of the theory 
and the model: Clauset, 1982; Clauset and Gaynor, in preparation; 
Clauset and Gaynor, 1982a, b. 



The first diagram (Figure i) illustrates the tasic f^edlDack 
structure for an effective school. .[4] 
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Figure 1. ^he Basic Feedt>ack Structure in an Effective; School 



[4] In the f/Lgure, the plus anl minus signs on ^he links between 
variables! indicate the nature of the relationship (i.e*, direct or 
inverse) J For example, a plus sign between achievement and 
motivation means that as achievement increases, motivation will 
alio increase and as' achievement decreases/^ motivation decreases. 
The minup sign between achievement amd perceived learning gap means 
that as; achievement falls^ the perceived learning gap becomes 
larger and achievement rises, the gap becomes smaller. The 
arrowheads on th^ links between variables indicate the direction of 
causation. ' » ' ' ; 



The causal relationship shown between achievement and instruction is 
'consistent with the BTES resear.ch in California on achievement, .learni-ng 
rate, and academic learning time (Fisher, et al. , 1978). [5] 
'•Appropriateness and Intensity of Instruction directly affects the amount 
bf engaged time. -It also affects student motivation as children compare 
the instruction they receive with that given to others and as, they ar^,^ 
affected directly by instruction which is more or less stimulating to 
them. Motivation to l^arn is,, in addition, influenced by the cllild's 
perception of the teacher's expectations fbr him and by his awareness of 
his achievement relative to grade level standards (cf. Kolesnik, 1978). 

- The level of appropriateness and intensity 'of instruction for a 
given achievement group is dependent on the amount of ■ time for 
instruction in tne subject, 'the teacher's effectiveness, and the amount 
of emphasis the teacher places on the group. Time for Instruction is a 
function of: (1) school policies for allocating time among subjects and 
for allocating time between instructional and non-instructional 
activities '(assemblies, lunch, recess, etcV) and (2) time the teacher 
must spend dealing with classroom behavior problems. Time spent on 
behavior problems depends on, the level of stucjent behavior in the classy 
the teacher's effectiveness, and the impact of the behavior of other 
students in the school. Teacher effectiveness refers to both the 
teacher's instructional and classroom ifenagement skills. Effectiveness 
is mediated by class size, altrtough more highly skilled teachers are 
less affected by larger and more able to take advantage of smaller class 
size^ 

The third component of appropriateness and intensity of instruction 
is the amount of emphasis a teacher gives to a particular achievement 
group. In. heterogeneous classes, this emphasis is a function of a 
teacher's desired emphasis and the competing demands of other groups. It 
is central to our theory of schooling that the perceived learning gap. 



[5] Partly for purposes of simplification, we chose in constructing the 
^ m.gxafel^ to assume equal •♦native learning ability" for all . children in 
tb§ ^simulated ^elementary school. Clearly, this is not, likely to be 
preibrisely true; however, a fundamental part of our thesis is that 
it i^S more true than would appear by deduction from the variance in 
actual achievement scores. ''An important purpose of our modeling 
^ effort was tc demonstrate theoretically that results very similar 
to those obtained historically in real schools (i.e., in terms of 
the divergence in achievement scores) could , be accounted for 
/ ' without assuming differences in native ability. What was assumed 
in the model is that students differed in their entry achievement 
(i.e.', "learning readiness"). We offered no interpretation with 
respect to the causes of those " initial * differences, neither 
hereditary" nor environmental. We have simply hypothesized, on the 
I .basis of what we believe to be reasonably compelling evidence, that 
many children are capable of learning far more in school than they 
do and that learning is importantly a direct effect of the 
' ]p' ' appropriateness and intensity of instruction (Bloom, 197^; Fisher, 
et al,, 1978). 



between teacher expectations and • the level of achievement is a major 
determinant. of teacher emphasis. A teacher will devote more emphasis to' 
a . particular achj^fivemepi group if the teacher perceives a gap ■ in 
achievement. If there is no gap between expectations . and' achievement , 
there will be no effort to increase the emphasis for a particular group. 
In the effective school, expectations are based solely on standards and 
not on pas€ achievement. Consequently, there is a sig^iificant gap in 
achievement for the initially low achievers and teachers want to place 
more .emphasis on these students to i^aise their achievement.' 

In systems terms, the basic driving force in the effective . schodl 
§tructyre is a negative feedback loop which operates to control the' 
level of student achievement by adjusting the appropriateness and 
intensity of instTruction. The goal of this control system ds to glose 
the discrepancy between teacher expectations (which are based-on fixed' 
standards for all children) and student achievement. • • ' 

The school which is ineffective for initially low-achievers is 
driven by a different causal configuration (Figure 2). T^iis difference 
may not be immediately apparent to the reader. Actually, the two 
•structures are identical except that teacher expectations are no longer 
based on a fixed set of achievement standards. 

For low achieving students in thi ineffective school, teachers' 
expectations respond directly to student achievement. This is the 
-essence of the ineffective' school. Studerfts who do poorly are expected 
to do poorly. Thus, frpm the teacher's perspective, 'there ^is no sense 
of a learning 'gap, no need to alter the nature of instruction, no 
ownership of an instructional problem, and no motivation to work at 
improving teaching skills. In essence, "The kid can't learn, or doesn't 
want to. It's his problem, not mine." 

The effect of directly linking teacher expectations and student 
achievement in the ineffective school is to collapse the negative 
feedback loop which operates in the effective sc{iool to control the 
level of student achievement. What now dominates the system is a 
positive feedback loop which reinforces existing achievement patterns. 
It works well enough for students, who are above average in achievement 
when they enter school. The system works to reinforce their motivation, 
their behavior, and ultimately their further achievement. However, for 
students who -enter- school with poor readiness skills, this "multiplier 
system" works to depress their motivation to leacn, tc reinforce 
dysfunctional behavior patterns of "acting out" or withdrawal, to reduce 
teacher emphasis on tnem, and to further diminish their future ' 
achievement. Thus, instruction becomes less appropriate and intense irf 
response to declining achievement. 
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The Theory and the Role of the Principal . It may have" occurred to 
the reader that the theory described above maJces^no reference to the 
school principal. In a sense, there appears to be no principal in the 
school. How can this be? 

There are two ideas behind this that are important to describe. The 
first is that there are two quite different ways to think about a school 
principal. One is to thinlc of the principal ^as a role-incumbenr , a 
member of the bureaucracy who appears on the table of organization of 
the school district. Ahother is to thinJc of the principalship as a set 
of functions to oe performed . The persdn is important only in terms"of 
what he or she does. • 
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In formulating tne theory, we have taken the second perspective. 
From this perspective, the principal is someone who performs leadership 
* functions^ However, the principal is not necessarily the only person 
who performs leader ^hip functions. Indeed, in viewing the school as an 
effective, operating institution, it is not essential bo describe "a 
principal" but only to understand -that certain leadership functions will 
be performed by someone or by some set of people. What, then, are these 
leadership functions? * 

. I*n Parsons ' criassiC terms, leadership is responsible for the 
organization's .performance of four imperative fu^nctions": integration, 
. pattern maintenance, adaptation, and goal attai'h.ment (Hills, 1968). In 
systems terms ,^ we have interpreted, that to mean th^t leadership is 
responsible for creating the structures that make possible the 
performance of essential functions. Thus, l«^adership is responsible, by 
definition, for implementing policies that, if necessary, alter- the 
structure of. the organization in ways., that enhance its ability to 
perfbrm these functions effectively and efficiently. 

This description of the role of leadership in an organization is, we. 
believe, consistent with traditional definitions of leadership in the 
management literature. Based on a cybernetic theory of management, the 
literature defines a control function for leadership, control is 
generally thought of, at least implicitly, as' a normative function 
coincidentfal with leadership. That is, 'it is seen as related to the 
"goals of the organization, whose attainment is the special 
responsibility of leadership. Consistent with the etymol^ical roots of 
the term "cybernetics" (from the Greek meaning fhelmsman") , control is a 
leadership function which involves ^ "steering" the orgajiization toward 
goals which are rooted in values. [6] 



[6] To avoid. • any possible mislinderstanding, it seems appropriate to 
establish certain demurrers' about the implications which* might be 
associated in the reader's mind about the word, "control." First, 
it carries no implications for us^ with respect to any particular 
sort of "leadership style." While for some, it may have 
connotation? of dxrectiveness . or authoritarianism, it can just as 
well imply participativeness of even the most collegial or communal ' 
proportions. Second, it begs the question of goal consensus. In 
the previous text, we used "goals'* 'in quotation marks in order to 
suggest the problematics associated with their conceptualization in 
an organizational context.' We recognize the political implications 
of goalsetting in practical social situations and the cui bono 
issues it raises (cf., Clegg and Dunkerley, 1980, chap. 8). 'In 
reality, the concept of control has no meaning except as it relates 
to goals, and the definition of organizational goals turns but 
empirically and ^^ilosophically to be highly problematic. ^However, 
this is an issue for another symposium. 

. .12 ■ 



11 



In tne diagram, of our theory of schooling (Figure 2), we have 
indicated several potential, points of intervention. These points 
represent^ entry points for exerting control over the teaching-learning 
process. Pressure for leadership action arises from a discrepancy' 
between the 'actual level of a variable, such as teacher expectations, 
and ajftore^ desirable level. This action results in the implementation 
of p^icies designed to bring the actual level of the variable closer? to 
the ^esired level. This process exemplifies the control function (or 
loop) in actual operation. . The implementation of such control processes 
implies the existence of policyTdriven control structure'^s through which 
the organization perceives and responds to undesirable discrepancie^s. 
Such control structures are made explicit by systiems analysts by 
diagramming ' negative feedbac)c loops which describe the 
perception-response dynamics. 

Implicit in^the fact that we have not shown trfese control loops fn 
the diagram is the idea that, in the ineffective school, the desired 
levols for teacher expectations, teacher emphasis, teacher 
effectiveness, student behavior, and time for instruction are the same 
as the current levels and there • is no active str^ucture for, perceiving 
and* responding to a problem, ihus, 'there no v discrepancy and no 
pressure from the leadership for change. The "principal" is there, but 
he or she is, de facto ,' entirely in agreement with the ^ta\us quo . In 
the improving school, there is a perceived discrepancy between actual 
and desired levels and the leadership exerts pressure for' change.. The 
control loop begins to operate. When we began our policy analysis of 
the relative effectiveness of interventions at these different points, 
our first step was to construct explicit control loops for each 
variable. 

Control has, in additioji -to the normative features \aescribea above, 
a generic aspect in social systems. Social systems can have "goals" 
which are not necessarily those of "leadership" but which are embedded 
in the sedimented historical structure of these systems. Such goaJ^ may 
often be covert (in the sense "of "unnoticed") but tlrey-are potentially" 
describable and can often , be powerful, even perverse. It is our 
theoretical' position_ that normative control (i.e., leadership) always 
' operates in an organizational context marked significantly by generic . 
control structures v^ich describe the historical tendencies of ' the 
social system to seeK certain natural goal states over time. 

An example of this conflict can be found in our theory of schooling. 
The level of student behavior is fundamentally driven by^ student 
motivation and perceptions of success in learning. It is generically 
controlled by these variables. Interventions by the principal or 
teachers to* change- behavior directly represent normative controls. ^We 
argue that normative interventions to improve behavior without improving 
instruction will be 'met witJ)i resistance '(by the generic control 
structure based on learning and motivation) and will, over time, have 
negligible, or even detrimeTntal, effects. 

Jt is our intent as we proceed with our theoretical 'work,. and ^th 
the empirical forays necessary to direct it, to do^ two interrelated 
things: (i) to describe the generic goal structures of schools and (2) 
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'to examine what policies might assist leadership in its quest to have 
normative ends prevail over undesirable generic ends. ^ Modally, 
"leadership" in schools signifies the school principal. Thus, as we 
examine policy options for transforming relatively ineffective schools 
into relatively effective ones, we discover that "the principal" is in 
the theory ^ after all, and that our policy analyses give shape and 
substance to the set of critical leadership functions he or she- 
performs. 

Froift ^ Theory to.;^imulation Model . In our research we were 
interested bot^h in developing a causal theory of schooling and in using 
the thebry to evaluate the liJcely * consequences of various school 
improvement policies. To accomplish the policy analysis, we translated 
our theory into a computer simulation model. [?] There are several 
advantages to computer simulation modeling. It enables one to test the 
internal consistency and robustness of one's theory ±n a manner which is 
impossible with mental models. Furthermore, the model allows one to 
sekrch systematically for important policy levets in the problem system 
and then to test a range of potential policy solutions for ameliorating 
the problem-. can evaluate the likely consequences, of different 

policies before investing large amounts of time, money and human 
resources on a * particular policy option in a real school or school 
district. 

The final advantage lies in the nature of the process. To write a 
set of mathematical equations for computer simulation, one must be 
explicit about one's assumptions and the relationships among system 
variables. The process draws on the existing research literature and on 
the experiental, qualitative knowledge of practitioners.^ It encourages 
dialogue and debate about assumptions and relationship?. The process 
is iterative. Trying to write equations to express a relationship may 
force one back to the literature for more information or \ may force one 
to reconceptualize the relationship completely. It is >this. circular 
process- of moving between the knowledge base, the theory, and the 
siTnulatior^ model that;^-J.eads to deeper understanding of trre problem and 
to a realization of "^those areas which require furt.her research. [8] 

* ' ■ ■ ■ 

Implications for Research . Having taken time to define our sense 
of the problem, our perspective, and the nature of our wofk, we now come 
to what we believe are the implications of our theoretical work for the 



[7] The method we used for building the computer model was System 
Dynamics. System Dynamics is a ^ particular form of systems analysis 
which was developed at M.I.T. during- the late I950's and has been 
refined in a- variety of applications over the last quarter century. 
It includes a set of tools and techniques for developing computer 
simulation models of dynamic causal structures. For a more detailed 
introduction to System Dynamics see Fotrester (1968) or Richardson 
& Pugh (1981). / , ■ ^ , 

[8] Despite its incorporation: of - computer technology, this approach 
appears analogous ^ conceptually"' to the so-called ".hermeneutic 
circle" (see, for example, Sergiovanni, "1982, pp. 75-76); .» 
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development of a sound research agenda with respect t6 improving schools 
for low-achieving children. Obviously, th^ nature of this agenda will 
suggest the nature of the measurement prbblems which are likely to be 
encountered. What is required is an ongoing program of iterative 
research and theory building* It is clearly npt^enough to go forward 
with pieces, even programs of -empirical research, without also investing 
<Ln ongoing ' programs of knowledge synthesis which • systematically, 
integrate accumulating research findings and -guide further empirical 
study. # 

Dynamic modeling is a knowledge synthesis and theory building Vool 
which requires several different types of decisions and information. It 
requires longitudinal data ai)out the problem behaviors of interest (such 
as patterns of student achievement) in order for the model builder to 
determine whether the model produces reasonable and realistic behaviors.* 
In the previous section on "Defining the Prpbiem," we described the 
kinds of data that we think are necessary for developing these reference 
behaviors. Modeling also requires decisions about which variables to 
include and which variables to leave out of the moc^el. In the field of 
systems analysis, these are called decisions about "model boundaries. 
It seems essential to us^ that all serious empirical- work; "be rooted, in- 
theories which are very clear about hypothesized model . boundaries .and 
causal path's (i.e. , what system dynsunicists call "causal-loops") . For 
example, it is well known to research methodologists that no meaningful 
statistical interpretation of data can be made without a -clearly 
explicated theoretical structure as a point of analytic reference 
(James, et al* . , 1982? Pedhazur, 1982, Chaps; 8-9). \^ ' 

><Sice the , model boundaries have been set, one must make decisions 
arfoulj which factors ^ to treat as constants and which ' to inciude 'as 
v^iables,. Model specification also requires information about< the 
precise nature- of the causal relationships among variables. Tofir' example, 
consider the relationships among teacher effectiveness,, time- available, 
for 'instruction, and engaged time. Engaged ^time will be .some ffciction -of 
the total time availc^Dle for instruction. This fraction cl^rly -depWnds 
on the teacher's effectiveness (skills Inediated' by class sizfe^-Vorkload 
pressures, and.committaent) . If -a teacher has "average" effectiveness and 
an "average" clans', wiat fraction of time available fcr'instrugtion is 
translated intb engaged time? If teacher effectiveness is 'above or 
below average f'^ojur will the .fraction change? Is the relationship between' 
effectiveness dnd fraction of time engaged a linear relationship? Are 
there maximum ^dr. minimum values (other than 0. and 1) for the fraction? 
.These are t^e kinds of questions modelers must ask of the litera^ture in 
seeking to translate general .theory into highly detailed computer; 
models. The BTES work and Stallings' research (c.f. Stallings, 1980) arl 
just beginijing to shed some light on these questions. 

Other questions revolve around setting initial parameters, such as 
the average length of the school day • or the "average amount of faculty 
releaS'e time for in-service, dt around determining the time it takes for 
one variable' to change in response to a change in a second variable 
(e.g., in an ineffective sch6ol, the time it takes for teacher^ 
expectations to move toward observed student achievement). 
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In the following paragraphs we describe some of the areas where we / 
felt the literature was not specific enough to provide detail^pd^ 
information about'^ the precise nature of the parameter values or the " 
causal relationsfiips among variables. These knowledge gaps imply the 
need for empirical research to provide further relevant information. 

• - 

There are five major areas in the literature where we' found 
deficiencies; 

o time, learning r and achievement 

o student motivation and behavior 

o teacher expectations 

o ^teacher decision-making for instruction 

o teacher effectiveness 

TIME, INSTRUCTION r AND ACHIEVEMENT 

We found in the literature a widespread recognition of the 
importance of time as an instructional variable. Time in our theory and 
.computer simulation model is the central medium of instruction. For 
example, the theory concerns itself with instructional vs. 
npn-instructional time and with the distribution of instructional time 
among subject areas and achievement groups. Skilled teachers are viewed 
as those who u§e time more effectively and efficiently. Disruptive 
student behavior takes time away from instruction. More ^motivated 
students are ..engaged in learning for higher proportions of time than 
lei^s Motivated students. The learning rate is a direct function of 
engaged time. 

A concern aftout time as an instructional variable has raised several 
issues for us. The first issue is a normative one. What are "normal" 
time allocations in elementary schools? How Jjong'is the average school 
day? What fraction of the school day is/* spent in the classroom? In 
non-instructional activities? How is time casually apportioned among 
subjects?' . ^ 

A second issue centers around the rela^onship between engaged time 
and learning rate. The behavior of our computer simulation model has 
raised questions about the nature, of this relationship. In the model, 
the learning rate and, therefore, student achievement is qtiite sensitive 
to policy interventions which affect time for instruction. It*" seems to , 
us that the sensitivity of th? learning rate to variations in engaged 
time* is / substantially more marked than our own experience in schools 
would suggest and thst real schools are substantially more "loosely 
coupled" with respect to time than the simulated school. We , have been 
puzzled by this, particularly since the , model behavior in other ways is 
consistent with common experience and research. 
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This apparent discrepancy between real behavior ajid model behavior 
raises for us a number of . questions which empirical research might seek 
to answer* What is the 'modal relationship in public schools between 
' clock time and engaged learning time? Is this relationship "loosely 
coupled" in the sense that variations in clock hours are compensated for 
in classrooms so that there is in real schools little correlation 
between the two? If so, how does this occur: i.e., what norms and 
structural dynamics govern the compensation mechanism?* To what eyctent ^ 
are stwdent turnover and absenteeism (variables not included in the 
model) factors in the compensation dynamic? How much engaged time (i.e., 
academic learning time) does it take to produce a '^normal" year's growth 
in reading or mathematics? To what extent is this relationship variate 
with respect to stable (i.e., unalterable) individual or cultural 
characteristics or to grade level? 

A third issue focuses on what might -be called a' saturation effect on 
engaged time. -Schools can attempt to raise significantly time on task, 
•but at what point does the law of diminishing returns begin to take 
effect so that further increases in time, on task do not lead to 
corresponding gains in engaged' time and learning? 

A -fourth^ issue relates to the impact of the student's past 
achievement on his/her present learning rate. We take' the theoretical 
position that the student's prror achievement can enhance or impede"^ 
present learning in the same subject and that the level of reading 
achievement can affect learning in other content areas as well. We have 
also incorporated the assumption that these effects are mediated by the 
quality and quantity of instruction the student receives. Whil^ there 
is some research oh this issue (c.f. Bloom, 1976), we believe more is 
needed to specify more precisely the nature of this relationship^ 

STU^T MOTIVATION ANfi BEHAVIOR 

Another area of concern is student motivation and behavior. In our 
theory, student motivation and behavior are important factors which 
enhance or constrain the learning process. Good behavior and high 
motivation increase time available for instruction and engaged time for 
learning. ^Disruptive behavior and low motivation work against teacher 
efforts to improve instruction. In the process of developing equations 
that linked motivation, behavior, and learning, we focused on questions 
such as: What are the determinants of behavior and motivation to learn? 
What are the. relative strengths of these determinants and do the 
determinants and their strengths vary with grade level? For example, 
how important are friendship patterns and ^eer relationships and how do 
these change developmen tally over time? [9] How fast does a student's 



[9] The School Effectiveness Model relates a student's behavior (in 
order of descending impact) to the motivation and behavior of his 
• peer group, the average level of behavior of the class, and the 
level of schoolwide behavior! Overall schoolwide behavior is most 
affected by older children and least affected by younger children. 
For a fuller discussion of dynamic relationships between academic 
achievement and peer choice, see Gaynor (n.d.). 
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motivation or behavior change in response to a change in one of these 
determinants? How does school-wide behavior affect the behavior of 
students in. a particular class? How does classroom behavior affect the 
school? Do the effects vary with grade level? Is. the effect of the 
behavior of a group of students on a class proportional to its relative 
size? Is there a "critical size" abo^e which a group^s disruptive 
behavior has a disproportionate effect? We fojind no clear consensus in 
the literature on these questions and we hope^ that more empirical 
research can be conducted in this area. ■ 

0 

A related issue th^t concerns behavior is whether it makes sense to 
View disruptive behavior as a cause of ineffective schooling or as a 
symptom. In our theory, behiv^ior is a manifestation of the 
problem — an outcome that can feed back into the system to have 
detrimental effects. Disruptive behavior is. driven by failure in "school. 
From this perspective, school improvement efforts that improve 
instruction are more effective than those that focus directly on 
behavior, alone, because they focus more directly on the underlying 
causes of the problem. However, some participants in various seminars we 
have given about school effectiveness have . suggested that improving^ 
discipline is a necessary first step. Some of these participants have 
been practitioners; others have been applied researchers. Perhaps there 
is a "breaking point" for bfehavior so that below that point one must 
first deal with behavior to reduce the chaos to a level where efforts to 
improve instruction can have some effect. Is this true? What is that 
breaking point? 

TEACHER EXPECTATIONS - ^ 

In the brief discussion above pf our theory of schooling, we 
emphasized that the crucial structurail difference between effective and 
ineffective schools is the manner in which teachers form expectations, 
Rist (1973) in his case study of an urban elementary school suggests 
that teachers with low expectations for children have a bias (conscious 
or unconscious) aga-inst them and, therefore, give them less emphasis and 
less appropriate and intense instruction even when there is no perceived 
learning gap between expectations and achievement. Is this a widespread 
.phenomenon? Indeed, has Rist described it accurately? 

Expectations can rise, as Mccormack-Larkin and Kritek (1982) have 
reported in Milwaukee. We hypothesized that teachers who perceive 
school-wide success in raising achievement will raise their expectations 
and give more weight to the belief that all but exceptional children can 
learn at or above grade level standards. What constitutes a perception 
of success for teachers? * How long does it take a change in achievement 
patterns to change teacher expectations? And what happens if teacher 
expectations are higher than student achievement and there is no 
* improvement over time? Do ejqDectations erode? What factors accelerate 
or retard this erosion? ^ 
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TEACHER DECISION MAKINQ FOR INSTRUCTION 

Teacher expectations are of little importance if they do not 
influence the decisions teachers make about the delivery of instruction 
to students. We have argued in our theory that teachers adjust their 
emphasis on students . and the appropriateness and intensity of 
instruction delivered in response to a perceived learning gap between 
expectations and achievement. This implies that past achievement is 
^.monitored and , that information^ about achievement is used in^diagnosis 
and planning. Is this, true?' Does teacher emphasis really vary with the 
perceived learning * gap*? How ' does this typically* transpire' 
phenomenologically? 

In a heterogeneous classroom, teacher decisions about the emphasis 
to be placed' on a given ^roup of students cannot be made without 
considering the needs 'of the other g^-pups in the class. No one group 
o can receive %ll of a teacher's 'attention. How do teachers normally 
divide attention among groups? Does it depend on group size? Perceived 
need? How do teachers react when the total demands for attention exceed 
their capacity? Who wins 'or loses? 

TEACHER EFFECTIVENESS 

The fifth and final area for further research focuses on the role of 
teacher effectiveness in promoting school improvement. We see teacher 
effectiveness andjbeacher emphasis andl time available for Instruction as 
the three determft^nts of appropriateness and intensity of instruction 
for a given gpoixp of students. Teacher effectiveness focu$es one's 
attention on understanding how teachers translate time available , into 
time on task. How efficient are teacher^s with average skills in 
accomplishing this? What it mean to speak of "average" skills? How 

does this* efficiency vary with changing levels of skill?* Does teacher 
effectiveness vary from group to group within a classroom? Why? If 
teacher effectiveness is thought of as the application of skills , what 
other factors mediate the relationship between skills and effectiveness? 

we believe specific studies are needed to address questions like the 
ones we have raised above. Some aspects of these studies may be 
expe^;-imental in nature but other aspects, especially .those which seek to ^ 
describe effects on process may require ethnographic approaches to data 
collection and interpretation. Efforts to integrate findijigs across 
disciplines can be expected to excacerbate epistemological strains 
between experimental and more phenomenologically rooted paradigms, we 
believe that it will be increasingly necessary to explore innovative 
methods for interpreting, displai^^ing, and examining inter-paradigmatic 
findings. 
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Utilizing "Alterable" Variables 

The third of ^the four questions* identified in the introduction to 
this paper concentrates on the link between theory and practice — the 
identification of those school process and climate variables which 
represent potent forces for change. Our policy analysis , and the work of 
others (c.f. Brookover, et al., 1979 and Edmonds, 1979) suggest that 
certain variables can have a significant impact ono» the level of . 
effectiveness in a particular school. 

In Figure Zf we indicated points of intervention in the system that 
through computer simulation poli9y analysis had demonstrated' potential 
for ameliorating the problem of ineffective .schools. From a systems 
perspective, each one of these points of intervention becomes the basis 
for a new negative feedback loop that can operate to control the level 
of the variable that is, to bring it in line with some desired value. 
As we mentioned before, the operation of a policy control loop requires 
that the goal be specified, that present conditions be moni^tored, and 
that actions be taken in response to a perceived discrepancy between the 

goal and the actual conditions. •• ' 

^> ^ • ■ ^ . 

For example, consider teacher skills. . To mount a program for 
improving teacher skills, one must first know the present level of skill 
in the faculty and the level to which the skills are to be raised. The 
magnitude of the discrepancy indicates the -type of intervention th^t 
might, be needed. One cannot begin designing implementation strategies 
until baseline data are gathered and goals are set. This is precisely 
the strategy adopted by the State of Connecticut in its School 
Effectiveness Project. ^ 

, The point we would like to emphasize herfe is that what variables one 
chooses to measure and the manner in which they are measured is directly 
dependent on the theory in wiiich these variables are imbedded and the 
level of specificity of that* theory. Our computer simulation model, the 
School Effectiveness 'Model, .represents a highly specified theory . of 
schooling. ^ In the process of building the model we were forced to think 
about how to quantify and operationalize different variables. The model 
shapes the ^ay we think about mea^suring school effectiveness. 

Implementing Change Programs 

Finally, there is in 'our judgment a vast and difficult domain of 
research which it seems crucial to. do, or at .least to begin to do at 
this time. Once the problem has been defined, the driving forces that 
generate the problem understood, and the points of intervention 
identified, one must grapple with how to .structure and implement an 
effective change program. Our work and the work of many others has 
focused on the nature of effective schooling and on the critical 
differences between effective and ineffeative schools. Our work has 
emphasized. in particular the effects of feedback and has examined the 
dynamic qualities of these two types of schools. What is .important 
about- the work that has been done, and about the research agenda 
discussed above, is that it poses practical directions for- improving 
schools. However, what it does not speak to are. the problems of 
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implementing the changes which are 'necessary if these new directions are 
to be achieved. 

» ■ 

We have already begun to discuss with a variety of individuals and 
, foundations the importance of studying recent and ongoing efforts at 
systematic, planned school improvement. We must have a growing theory 
of effective schooling, ^ one leased on the mutual reinforcement » of 
knowledge synthesis, theory building, and empirical research. We must, 
also have - a developing theory of the implementation of effective 
schooling, also based upon theory-guided research. Indeed, there can be 
no useful theory of effective schooling without a complementary theory 
of -the dynamics of organizational change by which effective processes 
can be 4impleme^nted in real schools. . 

^ t 

We see the basic structure of thi-s theory about the implementation 
of effective school programs as building upon the theory of schoolXng we 
have already developed. The points of contact are those points of 
. intervention in the system where the .principal or others might seelc to 
influence the causal structure affecting student achievement. In our 
initial round of policy analysis, we assumed (tentatively, for purposes 
of simplicity), that, an intervention, such as raising teacher 
expectations, would be successfully implemented. However, in reality, 
implementing school improvement policies has proven to be highly 
problematic (see, for example, Baldridge and Deal, 1975, Part ill; 
Berman and JJcLaughlin, 1978; Gaynor, ■ 1980; Goodlad, et al., 197-1; Smith 
and Keith-; 1971). " 

We would like to pursue further empirical research in schools that 
n?ve made systematic efforts to improve. This research would attempt to 
elucidate the web of variables (e*g., staff motivation for- change, 
conflict, staff workload, principal* skill and time, political support, 
and resources)^ that influence the outcomes of implementation. 

The proposed research would be informed by work already done ip 
trying to understand the dynamics of innovation (and stability) in 
public schools. ■ This work has developed some structural conceptions 
about ' the mutually reinforcing dynamics among external funding, 
leadership effectiveness, external linkage, ' and innovation. It has also 
explored the structural dynamics, especially those involving internal 
staff conflict and external political resistance,' which limit innovation 
and stimulate the discontinuation of innovations once implemented. This 
work has included an empirical study seeking to clarify processes 
associated with the- "life cycle" of a curriculum innovation in an 
elementary school (Gaynor, 1980) , a. study designed to examine 
empirically the phenomenology of the theoretical dynamics described in 
the Public School Change Model (Gaynor, ...i9'?9; 1981)'. 
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' Summary 

There.^has men historically in most schools a systematic difference 
in subsequent \ achievement between students who enter with average and 
low i'nltial levels of learning readiness. .Based on a comprehensive 
review of the \ extant literature on effective teaching and effective 
schooling, we have hypothesized that^this wieening gap is a function of^ 
the dynamic str,i\cture of the school. 

A computer Simulation model was described -which depicts a crucial 
structural difference between schools which . are effective and 
ineffective for injltially low-achieving children. Ineffective schools 
operate for all cftixdren as "multipliers" of existing differences. This 
comes about because ' the ^quality of instruction, /mediated largely by 
teacher expectations, varies directly with differences in student 
achievement, $he ^«sult is ' that high achievers are systematically 
reinforced to do bet\ter and low achievers are systematically reinforced 
to do worse. 

In contrast., schot^ls whiich are effective for initially low-achievin<^ 
stucjents operate for ihese children as "thermostat systems;" Teacherfe 
in effective scJfifoolS \def ine as problematic discrepancies between 
observeiS .student ^achievetjfnent and grade level standards. * Students whose 
achievement is below \ grade level statndards are defined as 
"under-achievers^ and efforts are made to improve tlie appropriateness 
and intensity of th^ir instruction.^ 



Based on an examinatioh of thfi substance of this perspective on 
effective and ineffective schooling, and consistent with the view that 
important problems , tend generally .to be generated by' the dyneunic 
properties of feedljac)c systems operating over time, some suggestions- 
were put forward" "with regard to a research agenda on effective 
schooling. These included general suggestions about the crucial need 
for longitudinal rjBsearch, theocy-guided research, and for ^mphasis on 
knowledge synthesis and theor^ formulation as well as specific 
» suggestions for research on subWantive variables and . relationships 
fUnong variables. We also emphasized the need for research on the 
organizational dynamics of implementing school improvement progrcuns. 
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